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METHOD AND APPARATUS FOR DETECTING REPETITIVE MOTION 
IN AN INTERLACED VIDEO SEQUENCE AND APPARATUS FOR 
PROCESSING INTERLACED VIDEO SIGNALS 



BACKGROUND OF THE INVENTION 

5 

CROSS-REFERENCES TO RELATED APPLICATIONS 

Applicants claim the benefit of U.S. Provisional 
Application No. 60/257,338 entitled ''Methods of Detecting 

10 Motion in an Interlaced Video Sequence Based on Logical 

Operation on the Linearly Scaled Motion Information and the 
Apparatus Therefor," filed December 20, 2000, U.S. Provisional 
Application No. 60/257,365 entitled ''Methods of Detecting 
Motion in an Interlaced Video Sequence Utilizing Region-Wise 

15 Motion and Apparatus" filed December 20, 2000, and U.S. 



M. Provisional Application No. 60/273,100 entitled ''Method of 

".W 

ry Detecting Repetitive Motion In An Interlaced Video Sequence 



and Apparatus Therefor," filed March 2, 2001, which 
applications are incorporated herein by reference. 

20 

FIELD OF THE INVENTION 



The invention lies in the signal processing field. More 
specifically, the invention pertains to a method of detecting 
25 motion in an interlaced video sequence. The invention is 
particularly applicable to the computation of a motion 
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decision value in an interlaced video signal and to a method 
of interpolating video signal information from existing video 
signal information. The invention also pertains to an 
apparatus for performing the method. 

5 

In the development of current Digital TV (DTV) systems, 
it is essential to employ video format conversion units due to 
the variety of video formats that are adopted in many 
different DTV standards worldwide. For instance, the ATSC DTV 
10 standard system of the North America adopted 1080x1920 
interlaced video, 720x1280 progressive video, 720x480 
interlaced and progressive video, and etc. as its standard 

iT| 

;.p video formats for digital TV broadcasting. Video format 

M. 

|,fj conversion, in this context, means to convert an incoming 

H. 15 video format to a specified output video format to properly 
fy present the video signal on a displaying device such as a 



monitor, FLCD, or a plasma display, which has a fixed 
resolution. Designing a good video format conversion system is 
of significant importance since it can directly affect the 
20 visual quality of the video of a DTV receiver. Fundamentally, 
the video format conversion operation requires advanced 
algorithms for multi-rate system design in which poly-phase 
filter design and interlaced to progressive scanning rate 
conversion or simply deinterlacing are of importance. 

25 
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Historically, deinterlacing algorithms were developed to 
enhance the video quality of NTSC/PAL TV receivers by reducing 
the intrinsic annoying artifacts of the interlaced video 
signal such as a serrate line observed when there is a motion 
5 between fields, line flickering, raster line visibility, and 
field flickering, where deinterlacing represents an operation 
that doubles the vertical scanning rate of the interlaced 
video signal. Besides, as mentioned above, the remarkable 
development of digital TV systems worldwide has raised the 
10 importance of the 3-D deinterlacing algorithms for the format 
conversion. For instance, it is highly demanded that 480 line 
interlaced video format needs to be up-converted to 720-line 



i:p progressive or to 1080-line interlaced format in the ATSC 

\,f\ system. In such case, it is well-known that pre-up-conversion 

M' 15 to 480-line progressive - which is deinterlacing, as will be 
9 

I'Li described in the following - provides more pleasing video 

Ui 

O quality than the direct up-conversion especially when a highly 
U 

elaborate 3-D deinterlacing algorithm is incorporated. 



20 The fundamental notion behind a 3-D deinterlacing 

algorithm based on motion detection is to cope with the 
theoretical limit of the sampling theorem embedded in the 
interlaced video sequence by properly detecting the point-wise 
motion of objects in the sequence. Some 3-D deinterlacing 

25 methods are found in the following prior art references: 
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[1] "Motion adaptive pro-scan converter with two 
dimensional contour enhancement," in. IEEE Transactions on 
Consumer Electronics, pp. 110-114, May 1990 by C. 
Markhauser . 

[2] Interlaced to progressive conversion apparatus 
and method using motion and spatial correlation, Young- 
Taeg Kim, U.S. Patent 5,943,099, Aug. 24, 1999. 



10 [3] Motion picture detecting method, Yong-Hun Cho, 

h U.S. Patent 5,959,681, Sep. 28, 1999. 

Q 
O 

ry 

The premise underlying the methods described in the 
IJl references [1],[2], and [3] is to estimate a motion decision 

M: 15 factor based on a frame difference signal in principle. 

Q 

I'U Although these methods utilize motion detection schemes to 

Q overcome the limit of the sampling theorem and as a 

consequence to provide better interpolation for the 
deinterlacing, there exists a theoretical limit in detecting 
20 real motion in the interlaced video sequence due to the 
intrinsic nature of an interlaced video sequence. To be 
specific, those motion detection methods will fail to 
detecting motion when fast repetitive motion is presented in a 
given interlaced video sequence. In other words, those methods 
25 will determine that no motion is present, where in fact fast 
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repetitive motion is present, and therefore perform a temporal 
interpolation. 

Repetitive motion can be easily observed when a pattern- 
like object is moving. This is illustrated in Fig. 1 where the 
shaded oval represents an object which moves up and down 
rapidly over a constant background, and where the object is 
spatio-temporally sampled in accordance with the interlaced 
sampling system. Observe that in this example a repetitive 
motion is presented during some time interval. In Fig. 1, the 
black circles represent the sampled lines whereas the lightly 
shaded circles represent the missed sampling lines that need 
to be interpolated. Assuming, for instance, the motion 
detection in this sequence is estimated by simply taking the 
field difference, no motion shall be detected at the point "b" 
in Fig. 1 since the sample values of "a" and "c" are from the 
constant background and, thus, the difference between them 
will be zero, meaning "no motion". Based on such "no motion" 
information, if we interpolate "b" with the sample value of 
"c", which is a typical case of motion detection based 3-D 
deinterlacing method, an artifact will appear since the 
resulting deinterlaced image will look like the background 
appears inside the object. If the object is composed of 
apparent signal levels from the signal level of the 
background, the artifact will be more visible. Hence, it is 
desirable to detect such motion and use a spatial 
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interpolation rather than the temporal interpolation in order 
to lessen such artifact. For this example, if we further 
consider the motion in the previous fields such as the 
difference signal denoted as "d" and "e" in Fig. 1, in which 
5 the difference signal "e" clearly indicates motion, and, thus, 
if we decide that there possibly exists fast repetitive motion 
around the point "b" the decision is made to use more spatial 
interpolation, so that the artifact can be lessened. 
Conversely, when certain motion is detected (at the point 
10 in Fig. 1) , propagating this motion information to the next 
associated points in the future fields for the motion 



Q detection is the fundamental notion of the method in this 

ru 

'nCi invention. 

m 

u 

M' 15 As we presented in the above, it is well studied that the 

p 

fU impact of applying a temporal interpolation based on a 

Ui 

misjudged motion detection method is a serious disparagement 



of the picture. An unacceptable amount of visually annoying 
artifacts can occur in the temporal interpolation if the 
,20 motion is not precisely detected. 



SUMMARY OF THE INVENTION 



It is accordingly an object of the invention to provide a 
25 method and an apparatus for detecting repetitive motion in an 
interlaced video sequence, which overcomes the above-mentioned 
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disadvantages of the heretofore-known devices and methods of 
this general type and which provide for a robust method of 
estimating a motion decision parameter which is associated 
with the point to point degree of motion including rapid and 
5 repetitive motion in the interlaced video sequence by further 
considering the motion information of the previous fields in a 
recursive fashion. It is another object of the invention to 
disclose a robust 3-D deinterlacing method and apparatus which 
utilizes the recursive motion decision parameter of the 
10 invention. 




provided, in accordance with the invention, a method of 



computing a motion decision value in a video signal processing 



With the foregoing and other objects in view there is 




15 system. The method comprises the following steps: 



sequence of fields; 



inputting a video signal with an interlaced video 



comparing mutually corresponding fields and defining 



a point-wise non-recursive motion decision parameter 



20 



indicating motion at a given point between a previous 



field and a next field in the video sequencer- 



computing a recursive motion decision parameter by 



combining the non-recursive motion decision parameter 



-8- 



with a motion decision parameter of at least one 
associated previous field; and 

outputting the recursive motion decision parameter 
for further processing in the video signal processing 
5 system. 



10 



In accordance with an added feature of the invention, the 
step of forming the point-wise motion decision parameter 
comprises computing 

f„{Uh) = lAdn{iM 



m 

ffl where fn(') is a point-wise motion detection signal, i and 

H= h define a spatial location of the respective video signal 
O 

15 value in a cartesian matrix, and Ik(') denotes a linearly 

-'"^ scaling function. 



In accordance with an additional feature of the 
invention, motion information of the associated previous 
20 fields is taken into account in defining a current motion 
defined by the recursive motion decision parameter. 

In accordance with another feature of the invention, the 
recursive motion decision parameter M^{i,h) is in the form of 
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M„iUh) = Fim„(i,h),M„.2(iM 

where F(') represents a monotonous function with respect 
to M„{i,h) and M ^_2(iji) having imposed thereon the following 
5 condition: 

min(m„ (i, h%M„_^(i, h)) < F{m„ (/, h\M (i, h)) < max{m„ (/, h),M„_^ (/, h)). 

In accordance with a further feature of the invention, a 

H< 10 non-recursive motion detection signal is computed from the 
O 

Q point-wise motion detection parameter by an equation selected 

9 

\U from the group consisting of 

M: 

in 

<^0,/i) = y:.(/,A)+min(X.,(/-l,AU_,(/ + l,/z)) 
Q Ui'h) = max(/„ {i,h),f„.M " ^AL-M + L^)) 

i 

where /„.iO denotes a motion detection signal delayed by 
one field, med{^) denotes a median operation, jnax(') denotes an 
20 operation to minimize an error from a false motion detection, 
and the indices i and h define a spatial location of the 
respective video signal value in a cartesian matrix. 
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There is also provided, in accordance with the invention, 
a method of processing interlaced video signals, which 
comprises : 

spatially interpolating a value of the video signal 
5 at a given location from a video signal of at least one 

adjacent location in a given video field; 

temporally interpolating the value of the video 
signal at the given location from a video signal at the 
same location in temporally adjacent video fields; 

Mi 

Q 

£3 10 forming a recursive motion decision value for the 

ru 

i,g same location as outlined above; and 
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ru 
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mixing an output signal for the video signal at the 
given location from the spatially interpolated signal and 
the temporally interpolated signal and weighting the 
15 output signal in accordance with the recursive motion 

decision value. 



In accordance with again an added feature of the 
invention, the motion decision value is varied between 0 and 1 
20 as a function of an estimate of the degree of motion at the 
given location and, upon estimating a high degree of motion, 
heavily weighting the output signal towards the spatially 
interpolated signal and, upon estimating a low degree of 
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motion, heavily weighting the output signal towards the 
temporally interpolated signal. 

In accordance with again an additional feature of the 
invention, the spatially interpolated signal is output as the 
output signal upon estimating a high degree of motion, and the 
temporally interpolated signal is output as the output signal 
upon estimating a low degree of motion. 

With the above and other objects in view there is also 
provided, in accordance with the invention, an apparatus for 
.computing a motion decision value in a video processing 
system. The apparatus comprises: 

an input for receiving a video signal with an 

interlaced video sequence of successive fields; 

a non-recursive motion detection unit connected to 
receive the video signal and to compute and output a non- 
recursive motion decision parameter defining a motion 
difference between a previous field and a next field of a 
current field to be deinterlaced; 

a recursive motion detection unit connected to 
receive the non-recursive motion decision parameter and 
configured to compute a recursive motion decision 
parameter by combining the non-recursive motion decision 
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parameter with a motion decision parameter of at least 
one associated previous field. 

In accordance with yet an added feature of the invention, 
the recursive motion detection unit is configured to take into 
account motion information of the associated previous fields 
in defining a current motion defined by the recursive motion 
decision parameter. 

The apparatus according to the invention is configured to 
carry out the method as outlined above. 

The apparatus for processing interlaced video signals 
according to the invention comprises: 

an input for receiving a video signal with an 
interlaced video sequence of fields; 

a spatial interpolator connected to said input and 
configured to spatially interpolate a value of the video 
signal at a given location from a video signal of at 
least one adjacent location in a given video field; 

a temporal interpolator connected to said input in 
parallel with said spatial interpolator for temporally 
interpolating the value of the video signal at the given 
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location from a video signal at the same location in 
temporally adjacent video fields; 

a processing apparatus as outlined above connected 
to said input and in parallel with said spatial 
interpolator and said temporal interpolator for forming a 
motion decision value for the same location; and 

a mixer connected to receive an output signal from 
each of said spatial interpolator, said temporal 
interpolator, and said computing apparatus, said mixer 
being configured to mix an output signal for the video 
signal at the given location from the spatially 
interpolated signal and the temporally interpolated 
signal in dependence on the recursive motion decision 
value output by said processing apparatus. 

Other features which are considered as characteristic for 
the invention are set forth in the appended claims. 

Although the invention is illustrated and described 
herein as embodied in a method and an apparatus for detecting 
repetitive motion in an interlaced video sequence and an 
apparatus for processing interlaced video signals, it is 
nevertheless not intended to be limited to the details shown, 
since various modifications and structural changes may be made 
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therein without departing from the spirit of the invention and 
within the scope and range of equivalents of the claims. 



The construction of the invention, however, together with 
5 additional objects and advantages thereof will be best 
understood from the following description of the specific 
embodiment when read in connection with the accompanying 
drawings . 



10 BRIEF DESCRIPTION OF THE DRAWINGS 

a 
o 

O Fig. 1 is a schematic illustrating fast motion in an 

W 

HQ interlaced video sequence; 

M' 
l.fl 

H' 15 Fig. 2 is a schematic view of a top field and a bottom 

Q 

i'U field of an interlaced video signal; 

Fig. 3 is a schematic diagram illustrating three 
successive fields of an interlaced video sequence; 



20 



Fig. 4 is a functional block diagram illustrating a 
recursive motion decision parameter calculation; 



Fig. 5 is a schematic diagram illustrating four 
25 successive fields; 
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Figs. 6-9 are block diagrams illustrating the 
computation of a non-recursive motion decision parameters 
m^{i,h) based on various alternative equations and function 
blocks; 

Figs. 10 - 13 are block diagrams illustrating the 
computation of a recursive motion decision parameter 

based on various alternative equations and function blocks, 
and each based on any of the recursive motion decision 
parameters m„(iyh) formed in accordance with Figs. 6-9; 

Fig. 14 is a block diagram of a device for converting an 
interlaced video input signal to a properly mixed deinterlaced 
video signal; and 

Fig. 15 is a block diagram of such a device incorporating 
any of the recursive motion detection units illustrated in 
Figs. 10-13. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In order to systematically describe the deinterlacing 
problem and the methods of the present invention, let 
denote the incoming interlaced video field at a time instant 
t = n and x„{vji) denote the associated value of the video signal 



1=! 
W 

=,0 
hi 

m 



at the geometrical location (v,/i) , where v represents vertical 

location and h represents horizontal location. By the 
definition of the interlaced video signal, it can be said that 
the signal values of x„ are available only for the even lines, 
5 i.e., v = 0,2,4,..., if is a top field. Similarly the signal 
values of x„ are available only for the odd lines of v (i.e., 
v = l,3,5,... ) if x„ is a bottom field. Conversely, the signal 
values of x^ are not available for odd lines if x^ is a top 
field signal and the signal values of x„ are not available for 
10 even lines if x„ is a bottom field. Pictorial description of 
the top and bottom fields of the interlaced video signal is 
illustrated in Fig. 2 where the image at t-m represents the 
top field and the image at t = m + \ represents the bottom field 



of an interlaced video sequence. 



Based upon the description of the interlaced video 
signal, the deinterlacing problem can be stated as a process 
to reconstruct or interpolate the unavailable signal values in 
each field. That is, the deinterlacing problem is to 
20 reconstruct the signal values of x^ at odd lines ( v = 1,3,5,... ) for 
the top field x^ and to reconstruct the signal values of x„ at 
even lines ( v = 0,2,4,... ) for the bottom field x„ . 
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For the simple description of the present invention, the 
deinterlacing problem will be simplified as a process which 
reconstructs or interpolates the unavailable signal value of 
x„ at the i^^ line where the signal values of the lines at 
/ ±l,i ±3,/±5,... are available. More simply, deinterlacing is to 
interpolate the value of x^{i,h) , which is not originally 
available. Note that, since x„_^ and have a different 
sampling phase from x„ , the signal values of (/, /i) and 
are available, which is the fundamental fact why motion 
detection can be incorporated with the deinterlacing problem. 
This relation is depicted in Fig. 3, where dotted lines (or, 
white circles) represent "no data available" and the solid 
lines (or, black circles) represent "available data". 

A motion decision (or, detection) parameter is estimated 
from the incoming interlaced video sequence and associated 
with the point-to-point degree of motion in the interlaced 
video sequence. The importance or the usefulness of the motion 
decision parameter can be best understood with reference to 
Fig. 3. Referring to Fig. 3, suppose that we have precise 
motion information when we want to interpolate x„{i,h) and 
suppose that no motion is encountered at the spatial location 
iUh) , then the best interpolation for x„(i,h) is to use the 
value of since the fact that no motion is introduced 
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between t-n-l and / = « + ! at the spatial location most 
likely implies that the value of x„(i,h) would be close to the 
value of x^_^(ijh) . The usage of the motion decision parameter of 
the present invention is also to utilize the motion 
information for deinterlacing to properly mix the temporal 
information, which will be disclosed later. 



In order to clarify the methods of this invention 
hereafter for the motion detection in the interlaced video 
sequence, terminologies for non-recursive motion detection 
method and a recursive motion detection method are described^ 
as follows. A non-recursive motion detection method will be 
referred to a motion detection method which estimates the 
motion decision parameter from a few number of fields such as 
x„_2 r ^n-i f f ^n+\ ' m„(i,h) denote as the non-recursive 

motion decision parameter of a typical non-recursive motion 
detection method for the interpolation of x„(i,h) which 
fundamentally utilizes the frame difference signal between x„_^ 
and x^^^ and/or further between and x^^^ . Without loss of 
generality, it can be said that the value of m„{i,h) is bounded 
as 0</M^(/,/i) <1 , where m^(i,h) = Q represents "no motion detected" 
and m^{i,h) = \ represents "motion detected". Then a motion 
adaptive deinterlacing method mixes a spatially interpolated 



value and a temporally interpolated value in accordance with 
the value of m^(i,h) . 

In contrast, a recursive motion detection method refers 
5 to a motion detection method which further utilizes the motion 
decision parameters of the previous fields. The functional 
block diagram of the recursive motion detection method of the 
present invention is disclosed in Fig. 4, where the non- 
recursive motion decision parameter, m^{i,h) , is computed from 
10 the fields, for instance, x^,, , x„ , and x^^.^ , where the recursive 



55 ^ 
motion decision parameter which will be denoted as M„(i,h) is \ 

!'! \ 

I If computed with the non-recursive motion decision parameter and \ 

the recursive motion decision parameters of the previous 

iji 

ii 

ru 



fields or frames which are stored in a memory, and where the 
15 computed current recursive motion decision parameter, A/„(/,/i), 



Y~, is then saved in the memory device. As disclosed m Fig. 4, 

■JSS' 

the recursive motion detection method of the invention does 
not necessarily limit the non-recursive motion detection 
method to a specific method. In other words, any existing non- 
20 recursive methods can be utilized with the current invention. 

In order to specifically describe the present invention 
for dealing with the fast repetitive motion in the interlaced 
video sequence, this invention further discloses the recursive 
25 motion decision parameter, M„{i,h), as in the form of 
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(1) 



where F{') represents a monotonous function with respect 
to M^{i,h) and Af„_2 0',/2)onto which the following condition is 
5 imposed: 

mm(m„{i,h),M„_^{iM < F{m„{i,h\M „_^{iM ^ m2Lx{m„{Uh\M (2) 

The fundamental notion behind the expression in Eq. (1) 
U: 10 is that the motion information of the associated previous 

b 

fields are taken into account when the current motion is being 
rU decided. In other words, the current motion decision will be 

M' made not only investigating the difference between the frames 



ii 



associated with current motion detection but also by looking 
15 at the previous-motion information. Conversely, when motion is 

ru 

detected at a certain time, this information will be 

O 

M' propagated over the time domain to secure the motion detection 

thereafter from the possible presence of fast repetitive 
motion. 



20 



As specific embodiments of the recursive motion decision 
parameter, M^{i,h), this invention discloses the specific 
functional forms of in the following. However, this does 

not necessarily limit the function to those forms. 



25 
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The first specific form of the recursive motion decision 
parameter, M^{i,h)t is disclosed as 



M^Uh) = F{m„{i,h),M„-2iiM 



where or is a pre-determined constant which controls the 
characteristics of the recursion and bounded as 0<«<1. For 
instance, no recursion is incorporated when a = 0 and the 
degree of recursion is increasing as the value of a approaches 



l!'} 10 1. Similar to the non-recursive motion decision parameter, the 

Q 

recursive motion decision parameter, M ^{iji) , is bounded as 



0<M„{i,h)<\, Also M„(i,h) = 0 shall imply "no motion", and 

M^(i,h) = l shall imply ^^motion". Note that M„ii,h) in Eq. (3) is 

M. 

Q in the form of an IIR filter structure, and that is the reason 

fU 

15 why it is referred to as the recursive motion decision 

O 

parameter. The interpretation of M^{i,h) given in Eq. (3) is 

that the motion in the previous fields is taken into 
consideration when the current motion is detected. The 
recursive nature of the Eq. (3) can be more easily understood 
20 by expressing it as follows: 



= m„ (/, /i) . (1 - a) + {m„_, (/, h)-{\-a) + M„_, h) a) a 
= m„ (/, h) (l-a) + (/, A) ■ (1 - or) • a + M„., (/, h) a' 
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and further it can be expressed as 



and so on. Clearly this shows the recursive nature of the 
motion decision parameter disclosed in Eq. (3) . 

The second specific form of the recursive motion decision 
parameter, M ^{iji) , is disclosed as 



which is to enforce the motion information when obvious 
motion is detected by a non-recursive motion detection method 
(i.e., when m^{i,h)>T)f and where T is a pre-determined 
positive constant. 

The third specific form of the recursive motion decision 
parameter, M„(/,/i), is disclosed as 




(4) 



= max(w„ 0, h), m„ (/, h) (\-a) + M„_^ (/, h) • a) 



(5) 
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where max(-) represents a two input function which outputs 
the greater value of its input. The functionality behind the 
Eq. (5) is to increase the certainty of detecting motion by 
imposing the greater motion decision value of a non- recursive 
method and the recursive method to M„ (/,/?) . 

The fourth specific form of the recursive motion decision 
parameter, A/„(/,A), is disclosed as 

M„{i,h) = F{m„{Uh%M^_,{i,h)) 

which is to mix the value of m„{i,h) and 
m„{i,h) {\-a) + M„^2i^,h) a depending on the value of m^{i,h) . That 
is, the value of M^{i,h) in Eq. (6) approaches m^{i,h) as m^{i,h) 
approaches 1 ("motion") and it approaches the value of 
m^(i,h)'(\-a) + M„_2Q,h)'a as m^(J,h) approaches 0 ("no motion") . 

So far, recursive motion detection methods of the 
invention have been disclosed as expressed in Eq. (1) through 
Eq. (6) . As mentioned earlier, the methods of the current 
invention do not limit the non-recursive methods of motion 
detection to a specific method. However, in order for the 
proper embodiments of the current invention, some non- 



recursive methods of computing motion decision parameter, 
m^{i,h) , are presented in the following next. 

First, the frame difference signal D^is computed by 
5 taking the difference between the fields in one frame interval 
as 



10 

O 
fli 

! PI 



ry 15 
1:3 



which associates with the scene change occurred between 
the fields ;c„^., and ;c„_, . The frame difference signal is then low 
pass filtered as 

where XPF(-) represents a low pass filtering process over 
the input video signal. The My.N kernel, fK^^^ , in general, 
of the low pass filter, LPF{) , can be expressed as 



20 



12 



^^21 ^^22 



W. 



'2U 



^A/1 >*'a/2 



MN 
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where {Wu>' "y^MN) represents a set of predetermined 
coefficients. It should be mentioned that the characteristic 
of the LPF() can be all pass filter depending on the choice of 
the kernel Wj^^i^ . That is, if the kernel is set as M = N = \ 
5 and =1, the LPF() becomes the all pass filter and, thus, 
=D„ , which is equivalent to have no spatial low pass 
filtering. 



Next, a point-wise motion detection signal is computed as 




10 



fShh) = lMiM 



(7) 



i s 

::b: 



::s: 



in 



where /j^(-) denotes a linearly scaling function represented 



■5 



b 
m 

o 



15 



as 




I if 




in which K is a positive constant value. 



20 



The simplest non-recursive method of estimating the 



motion decision parameter is to use the point-wise motion 



detection directly as the motion detection signal, i.e.. 



(8) 
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More elaborate methods of non-recursively computing the 
motion decision parameter will be outlined in the following. 

Next the region-wise motion detection signal is computed 
from the point-wise motion detection signal as 

= Ui,h) + min(/„.,0 - 1, + 1,/))) ( 9 ) 

where denotes the one field delayed motion detection 

signal described in Eq. (7) . 

Or, as an alternative method, we define the region-wise 
motion detection signal as 

= + (10) 

where med{') denotes a median operation. Or, in order to 
minimize the error from a false motion detection, we define 
the region-wise motion detection signal as 

Mhh) = max{f„{iMfn-xii-mJn^^ii + hh)). (11) 

Finally, the region-wise motion detection signal is low 
pass filtered and forms the non-recursive motion decision 
parameter as 
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b d 



(12) 



p=—a q-~c 



where a,b,c,d>0 and a^^ represents a set of normalized 



b d 



(i.e., XS^p.?"^ ^ predetermined coefficients of an low pass 



5 filter. For instance, the kernel of the low pass filter used 
in Eq. (12) can be 



o 
m 

i.ri 

M' 10 

i,ri 

i! 

]■* 

o 
m 
w 



■ 0 1/8 0 
1/8 4/8 1/8 
0 1/8 0 



By adjusting the coefficients of the low pass filter 
given in Eq. (12), various different characteristics of the 
motion detection can be realized. This also includes the all 
pass filter by having 



15 



0 0 0 
0 1 0 
0 0 0 



which is equivalent to having no low pass filtering. 



Fig. 6 is a block diagram to compute the non-recursive 
20 motion decision parameter m„(i,h) as described in Eq. (8) . 
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Fig. 7 is a block diagram to compute the non-recursive 
motion decision parameter m^{i,h) as described in Eq. (9) . 



Fig. 8 is a block diagram to compute the non-recursive 
5 motion decision parameter m^{i,h) as described in Eq. (10) . 



Fig. 9 is a block diagram to compute the non-recursive 
motion decision parameter m^{i^h) as described in Eq. (11) . 



1,^, 10 Referring now to Fig. 6-9, there is illustrated process 

b 

l!5 t>y which the non-recursive motion decision parameter mn(i,h) 

^\ 

ry can be estimated. Fundamentally, mn(i,h) is estimated from the 

jiii: incoming interlaced video sequence and associated with the 

in 

i\ point-to-point degree of motion in the interlaced video 

C5 15 sequence. The importance or the usefulness of estimating 

::{ I 

Ui mn(i,h) can be easily understood from Figs. 3 and 5. Suppose 

o 

H' that precise motion detection information is available when we 

interpolate Kn(i,h) and suppose there is no motion at the 
spatial location drh) , then the best interpolation for Xn(i,h) 
20 is to use the value of Xn-i(ifh). This follows logically from 
the fact that no motion is introduced between t = n - 1 and t 
= n + 1 at the spatial location (i,h), which very strongly 
implies that the value of Xn(ifh) would be close to the value 
of Xn>i (i,h) . 

25 
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# • 

First, the frame difference signal is computed by 
taking the difference between the fields in one frame interval 
as ^„ =|jc„+i -^„_il which associates with the scene change that 
occurred between the fields Xn+i and x„-i. The frame difference 
5 signal is then low pass filtered to form d„ =LPF(D„) where 
LPF{-) represents a low pass filtering process over the input 
video signal. The M x N kernel, WMxNf in general, of the low 
pass filter LPF{-)/ can be expressed as 



O 

ry 
m 



ru 
o 



10 



W = 



^21 ^ 



22 



W. 



2N 



W 



MN . 



where [wn, u^n) represents a set of predetermined 
coefficients. It should be mentioned that the characteristic 
of the LPF (•) can be all-pass filter depending on the choice 
15 of the kernel ^mxn- That is, if the kernel is set as M = W = 
1, and f/ii=l, the LPF(-) becomes an all-pass filter and, thus, 
d„ = Dn. 



Next, a point-wise motion detection signal is computed as 
20 f„={iyh)=lf({d^{i,h)) where /^^.Q denotes a linearly scaling function 

/ \ f 1. if y^^ 1 . . 

represented as lK(y} = i \f with K being a positive 

[y/ K, otherwise} 

constant . 
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As illustrated in Fig. 6, the function unit Bl may at 
this time output the non-recursively calculated motion 
decision parameter m„(i,h) for further use in the system. In a 
refined embodiment of the invention, as illustrated in Fig. 7, 
5 the function unit B2 may next compute the region-wise motion 
detection signal from the point-wise motion detection signal 
as defined in Eq. (9) - (/, /i ) + min(/„_, (i 0 + 1, /i)) , 
where /„_iO denotes the one field delayed motion detection 
signal described in (7) . The alternative method is illustrated 

|.A 10 in Fig. 8 with the function unit 83, where we define the 

Q 

region-wise motion detection signal as defined in Eq. (10) - 



m 

ii 



rU (pn{i,h)^rned(fXi,h)j„_X^-\,h\f^_Xi'^\M^ where med{) denotes a median 

operation. Or, in order to minimize the error from a false 
motion detection, we define the region-wise motion detection 
15" signal as illustrated in the function block 34 in Fig. 9 as 
;i| defined in Eq. (11) - (/,/i) = max(/„(i,/i), /„_,(/ -1,4 /,,,(/ + . 

Finally, the region-wise motion detection signal is low 
pass filtered to form the non-recursively computed motion 
20 decision parameter m„{i,h) of Eq. (12) . 

The various methods were disclosed in the preceding text, 
of computing recursive motion decision parameter for 3-D 
deinterlacing combined with non-recursive motion detection 
25 methods described in Eq. (8) through (11) . In the following. 
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we describe various embodiments of the recursive motion 
detection methods of the current invention with reference to 
Figs. 10-13. 



10 



I. J, 



Fig. 10 discloses the embodiment of the recursive motion 
detection method based on the non-recursive motion decision 
parameter disclose in Eq. (3), where the non-recursive motion 
detection unit can be either the block Bl, B2, 83, or B4 in 
Fig. 6, Fig. 7, Fig. 8, and Fig. 9, respectively. 



Fig. 11 discloses the embodiment of the recursive motion 

Q 

Q detection method based on the non-recursive motion decision 

ru 

parameter disclose in Eq, (4), where the non-recursive motion 
iri detection unit can be either the block Bl, B2, B3, or B4 in 

15 Fig. 6, Fig. 7, Fig. 8, and Fig. 9, respectively. 

O 

ru 



Fig. 12 discloses the embodiment of the recursive motion 
detection method based on the non-recursive motion decision 
parameter disclose in Eq. (5) , where the non-recursive motion 
20 detection unit can be either the block Bl, B2, B3, or B4 in 
Fig. 6, Fig. 7, Fig. 8, and Fig, 9, respectively. 

Fig. 13 discloses the embodiment of the recursive motion 
detection method based on the non-recursive motion decision 
25 parameter disclose in Eq. (6), where the non-recursive motion 
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detection unit can be either the block Bl, B2, B3, or B4 in 
Fig. 6, Fig. 1, Fig. 8, and Fig. 9, respectively. 

In the following, the 3-D deinterlacing method of the 
present invention is disclosed based on the recursive motion 
detection method disclosed above. 

The computed recursive motion decision parameter M^(i,h) 
is then used to mix a spatially interpolated signal and a 
temporally interpolated signal. Fig. 14 discloses the 
embodiment of the present invention for interpolating the 
value of x^{ifh) given the interlaced video sequence based on 
the recursive motion decision parameter disclosed in Eq. (1), 
which comprises a spatial interpolator 3, a temporal 
interpolator 4, a motion decision processor 5, and a mixer 6. 
The spatial interpolator spatially interpolates the value of 
x„(i,h) by using a predetermined algorithm. The temporal 

interpolator temporally interpolates the value of x^Q^h) by 
using a predetermined algorithm. The recursive motion 
detection unit computes the recursive motion decision 
parameter, M„(i,h), which represents the degree of the motion 
at the interpolation location {i,h) . The mixer mixes the output 
signal of the spatial interpolator and the output signal of 
the temporal interpolator in accordance with the recursive 
motion decision value. Letting x'„(iji) and x'„{i,h) as the output 
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signal of the spatial interpolator and the output signal of 
the temporal interpolator, the output signal of the mixer, or, 

the interpolated signal is represented as 



Note that x„{i,h) = x'„(i,h) when M„(i,h)~0 (no motion) and x^(i,h) 
= x'^{v,h) when M„(i,h) = l (motion). 

In Fig. 14, it does not matter what specific spatial 
interpolating algorithm and a temporal interpolating algorithm 
are used for the spatial interpolator and the temporal 
interpolator, respectively. Some examples of the spatially 
interpolated signal jc^(v,/i) are 



which corresponds to a method known as line doubling. 
Also some examples of temporally interpolated signal x'„{Vfh) 



x„{Uh) = {\-M^{iM<{Uh) + M,{Uh)x:{Uh). 



x:(i,h) = {x„(i-lh) + x„{i + lh)y2, 



which corresponds to a line average and 
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and 
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Fig. 15 discloses the embodiment of the present invention 
for interpolating the value of x„(i,h) given the interlaced 
video sequence based on the specific recursive motion decision 
parameter disclosed in Eq, (3) through Eq. (6), where the 
recursive motion detection unit can be either the block CI, 
C2, C3, or C4 in Fig. 10, Fig. 11, Fig., 12, and Fig. 13, 
respectively. 



-35- 



